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Abstract 

Aerospace systems are developed similarly to other large-scale systems through a series of reviews, where 
designs are modified as system requirements are refined. For space-based systems few are built and placed into 
service — these research vehicles have limited historical experience to draw from and formidable reliability and 
safety requirements, due to the remote and severe environment of space. Aeronautical systems have similar 
reliability and safety requirements, and while these systems may have historical information to access, commercial 
and military systems require longevity under a range of operational conditions and applied loads. Historically, the 
design of aerospace systems, particularly the selection of sensors, is based on the requirements for control and 
performance rather than on health assessment needs. Furthermore, the safety and reliability requirements are met 
through sensor suite augmentation in an ad hoc, heuristic manner, rather than any systematic approach. A review of 
the current sensor selection practice within and outside of the aerospace community was conducted and a sensor 
selection architecture is proposed that will provide a justifiable, defendable sensor suite to address system health 
assessment requirements. 


I. Introduction 

Aerospace systems are developed similarly to other large-scale systems through a series of design reviews. At 
each design stage, decisions and trades are made about all aspects of the vehicle: hardware, software, functionality, 
operations, etc. Each component, including sensors, must “buy its way onboard.” The costs and risks associated with 
flight dictate that unnecessary components be filtered from the design; therefore, all components must justify their 
existence and purpose onboard the vehicle. To achieve this, a component must satisfy a quantifiable and verifiable 
set of requirements that will demonstrate its benefit and purpose to the system. In addition, any risk to the system 
must also be identified. 

Traditionally, in flight systems, sensors are primarily selected through an ad hoc heuristic process, where 
various domain groups are polled as to what sensors they require in the system. Although safety and reliability are 
just as important as system performance, the sensors are primarily selected based on control requirements and 
performance assessment, rather than on health monitoring and management. To be incorporated into the design 
process, accepted methodologies and procedures must be established that provide justification of the selected 
sensors within the constraints of the vehicle requirements. To precisely quantify the benefits of sensors in flight 
systems, heuristic sensor selection approaches must give way to systematic techniques that are based on accepted 
selection criteria and performance metrics that measure their value to the system. 

Therefore, substantial benefit to the design and development process can be realized by utilizing a sensor 
selection methodology, specifically directed toward system health assessment. The purpose of this study is to review 
the current state of sensor selection within and outside of the aerospace system community and to propose a sensor 
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selection architecture that will provide justification of sensor inclusion based on the net benefit from the health 
assessment perspective. This review is not intended to be encompassing in terms of directly comparing the various 
techniques against each other and assessing their performance. Rather, the review is based on categorizing salient 
features of these approaches and the desirability of certain features when they are applied to aerospace flight 
systems. 

The paper is organized as follows. Sections II and III will review and discuss various techniques used 
historically for sensor selection. Section IV will discuss in detail the proposed sensor selection architecture 
originally designed to address health assessment for propulsion systems. Finally we will conclude with a discussion 
of proposed research areas that will advance the capabilities and credibility of the sensor selection process for 
aerospace diagnostic applications. A design example will be included in the appendix that will demonstrate an 
application of the proposed sensor selection technique. 

II. An Overview of the Sensor Selection Process 

A number of sensor selection approaches, applied to a variety of systems and for a variety of purposes, have 
been developed over the years. For the methods described in this review, the sensor selection process can be divided 
into two parts: an evaluation module and an optimization module. The evaluation module generates a performance 
metric for the sensor suite or network under review, and the optimization module searches through the space of all 
possible sensor suites and selects additional suites for further evaluation. The process is continued until the “best” or 
the optimum sensor suite is obtained. 

A. Sensor Selection Evaluation 

The goal of the sensor selection process is to provide a suite of sensors that fulfill specified performance 
requirements within a set of system constraints. These performance requirements are defined as the Figures of Merit 
(FOMs) of the system, and considerable research has focused on how to represent these FOMs in algorithmic form. 
The applied set of FOMs should reflect the overall goal of the selection process. The following list of general FOM 
categories was selected after reviewing the available literature and research. 

• Observability — This category considers how well the sensor suite will provide information about the given 
system process, which parameters that are directly observed, and which parameters can be inferred. 

• Sensor Reliability/Sensor Fault Robustness — This category addresses sensor reliabilities and how sensor 
availability impacts the overall sensor suite performance. 

• Fault Detectability/Fault Discriminability — This category specifically addresses whether the sensor suite can 
detect and discriminate system failures. 

• Cost — This category can include development, purchase, and maintenance costs for the sensors as well as 
resource and communication costs. 

These FOMs will be discussed in greater detail in section III, where we will identify specific methodologies and 
highlight examples of research that define them. 

B. Optimization Processes 

Analysis has shown that general sensor selection problems addressing diagnosability, or observability, are NP- 
complete and are therefore computationally intractable (ref. 1). These types of problems require fast approximate 
search solutions in order to generate acceptable results in reasonable time. Where possible, Brute Force or 
Exhaustive Search methods are preferable, but they examine all possible candidates and as systems become more 
complex, the time involved grows prohibitive. Therefore, methods have been developed which attempt to refine the 
searches through determination of potential candidates in order to find the optimal or near-optimal solution. 
Regardless of the selection strategy, sensor selection is based on a set of criteria called the objective function which 
is an algorithmic representation of established FOMs and system constraints. 

Many techniques have been developed for general optimized solution searches. These techniques range from 
applying constraints on the objective function to streamline the optimization process (ref. 2) to applying advanced 
artificial analysis techniques such as genetic algorithms and simulated annealing algorithms (ref. 3). Unique 
optimization techniques have been proposed such as particle swarm optimization (ref. 4) and cutting and surrogate 
constraint analysis (ref. 5). The point of this section is not to review each search solution or optimization technique 
but to identify the variety of algorithms available. Optimization techniques are well documented (ref. 6). Table 1 
reports the algorithms encountered in this literature review and references that utilized them. 


NAS A/TM— 2007-2 14822 


2 



TABLE L— SENSOR SELECTION OPTIMIZATION TECHNIQUES ENCOUNTERED 
AND ASSOCIATED REFERENCES 


Optimization technique 

Researcher 

Constraint-Based Search 

Narasimhan (ref. 7), Mushini (ref. 8) 

Exhaustive/Brute Force Search 

Debouk (ref. 2), Mushini (ref. 8), Narasimhan (ref. 7), Madron 
(ref. 9), Worden (ref. 3) 

Genetic Algorithms 

Musulin (ref. 10), Mushini (ref. 8), Spanache (ref. 11), Sen 
(ref. 12), Santi (ref. 13), Worden (ref. 3) 

Particle Swarm Optimization 

Zhang (ref. 14) 

Graph Theory — Spanning Trees and Cutsets 

Ali (refs. 15 and 16), Bagajewicz (refs. 17 and 18) 

Cutting and Surrogate Constraint Analysis 

Azam (ref. 1 9) 

Mixed Linear Integer Programming 

Bagajewicz (refs. 20, 21, and 22), Chmielewski (ref. 23) 

Simulated Annealing Algorithm 

Worden (ref. 3) 


III. Historical Review of Sensor Selection Processes 

The background for the selection of the FOMs presented in section II is described below. These metrics are 
based on qualitative as well as quantitative methods and derived from a number of disciplines such as controls 
theory, estimation theory, data reliability and failure analysis. The subsequent subsections will review some of the 
research development that has been done in each of the first three categories: observability, sensor reliability/sensor 
fault robustness and fault detectability/fault discrimination. The fourth category, cost, is a general penalty-driven 
FOM grouping and is very straightforward conceptually and in implementation. There is overlap between the 
arbitrarily defined categories, and the final sensor selection FOM may be a combination of performance and cost 
metrics from any or all of them. 

A. Observability 

Of ultimate importance for any sensor suite is the concept of observability. Basically, observability is the 
capacity of the sensor network to provide information about the state parameters deemed important for performance 
monitoring, health assessment, and/or control of the system. This information may be provided as direct 
measurements of the system parameters or as in data reconciliation , the reconstruction of unobservable system 
parameters based upon observable or sensed ones. Approaches vary in the determination of the observability of the 
sensor network. Some treat observability as a true or false property, while others devised more sophisticated 
schemes to display the degree of observability. 

Some approaches used graph-based analysis for which the structural information of a system is represented by a 
directed graph, called a digraph. Observability is then based on analysis of cutsets and cycles generated from the 
digraph. Kretsovalis (ref. 24) proposed two graphically-based algorithms for classification of observable and 
redundant variables. Luong (ref. 25) also used graph-based analysis to establish an incidence matrix, relating the 
process relationships to the state variables qualitatively. Decomposition of this incidence matrix using a Gauss- 
Jordan elimination process identified whether an unmeasured variable was observable and whether a measured 
variable was redundant. Using graph theory and cutsets, Bagajewicz (ref. 18) defined the degree of observability and 
degree of redundancy for a sensor network. For an unmeasured variable, the degree of observability is the maximum 
number of sensors that can be eliminated with the variable still observable; for measured variables, the degree of 
redundancy is the maximum number of sensors that can be eliminated and the measurement remains redundant. 
These concepts were combined to define a degree to which system variables can be estimated by the measurement 
network. 

Other approaches attempted to define the degree of observability by analyzing the numerical relationships of the 
system process. By taking advantage of linear state space system theory, observability and controllability can be 
represented in matrix form. The system is observable or controllable if the observability or controllability matrices 
are of full rank or nonsingular. These represent binary conditions as to whether the system is observable/controllable 
or not. Dochain (ref. 26) considered both the rank and condition number of the observability matrix for an optimal 
sensor placement within a bioreactor system. The Gramian of both the observability and controllability matrices 
have been used by researchers (refs. 27 and 28) to develop scalar metrics that indicated the degree of each respective 
property. 

A large amount of research has been devoted to control design and actuator/sensor placement locations, 
particularly in structural-type problems. Papadopoulos (ref. 29) investigated a number of quantitative techniques 
used to optimize structural sensor placement to select sensor locations that would observe the maximum amount of 
system information, in this case energy. Each technique attempted to characterize the amount of energy the sensors 
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would observe and eliminate potential sensor locations that had minimal observability. Hac (ref. 30) proposed a 
methodology that incorporated eigenvalues from both the observability and controllability matrices to select sensor 
locations for control of flexible structures. 

Several researchers used the predictive error of a given sensor suite relative to a specific reconciliation 
technique as the performance metric. After performing matrix decomposition on the linear system model 
coefficients, Madron (ref. 9) proposed a simple local optimization of this baseline set of sensors by computing the 
prediction precision of unmeasured required state parameters. The baseline sensor suite performance was compared 
to suite variations where a single sensor was removed and the objective function recomputed. Chmielewski (ref. 23) 
developed the framework for the justification and incorporation of the error covariance matrices for sensor suite 
performance properties and demonstrated how that could be optimized with other suite performance constraints 
using mixed linear integer programming techniques. Musulin (ref. 10) proposed a sensor selection process that 
maximized a Kalman Filter performance, via the error covariance matrix diagonal terms. Musulin pointed out that 
although the objective is to maximize the performance of the filter for all variables, maximizing the performance for 
one particular variable may not be compatible with maximizing it for others, resulting in a conflict. Therefore they 
proposed two possible approaches; one method would select the variable with the lowest performance value. The 
second approach would evaluate the performance of each variable relative to an “ideal” sensor suite performance, 
combining into an overall performance metric. Mushini (ref. 8) used a similar approach of maximizing a Kalman 
Filter performance for an aircraft gas turbine engine. The metric was defined as the summation of the error 
covariance matrix diagonal terms normalized to the reference or ideal system performance. 

B. Sensor Reliability/Sensor Fault Robustness 

Regardless of the purpose of the sensor network, consideration must be made for the availability of the sensor 
signal information when it is required. Sensor faults can be common, and the potential for interruption of 
information flow and how that is to be handled is an important consideration in the sensor network design analysis. 

Luong (ref. 25) investigated sensor network reliability from a control perspective that the reliability of an 
instrumentation system is the probability that information required for control are available through measurements 
or deduction during some time period. Based on that idea, an overall sensor network reliability metric was defined 
as an expression of the individual sensor reliabilities. Sensor reliabilities were expressed as a function of time; 
therefore, the integration of the time profile for each sensor network was used as a quantitative comparison metric 
between competing networks. 

Ali (ref. 15) introduced the concept of reliability of the estimation of variables, which relates the sensor 
reliability with its availability to provide system state estimations. The focus of the research was for a given sensor 
network, how many different ways can a variable be estimated and if sensors fail, can a variable still be estimated. 
The network objective function was defined as the minimum product of the input sensor reliabilities for each 
estimated parameter. The initial processing routine involved graph theory concepts of chords, cutsets, and spanning 
trees to establish sets of estimated variables and their associated sensors used to observe the variable. This objective 
function was extended to include hardware redundancy (measuring a variable using more than one sensor) and 
spatial redundancy (more than one way of estimating a variable) (ref. 16). Within the graph theory algorithm, the 
sensor network can be optimized with respect to only a single criterion; therefore, this objective function was 
adapted for processing with genetic algorithms (ref. 12) and mixed- integer nonlinear program (MINLP) (ref. 20) 
which enabled it to be evaluated with other performance metrics, such as sensor costs. 

Another consideration of the sensor network is the performance in the presence of sensor faults. Sensor 
networks must be robust to sensor faults, both in being able to perform adequately with a faulty sensor within the 
network and being able to continue performance adequately if a sensor is removed. Bagajewicz (ref. 17) proposed a 
set of performance metrics for a sensor suite that defined the ability of the sensor network to detect sensor faults 
(error detectability) and handle sensor faults relative to reconciliation precision performance (availability — precision 
after failed sensor removed, and resilience — precision in the presence of a sensor fault). Bagajewicz (ref. 21) 
extended this research from the graphical tree-search-solution procedure to the explicit MINLP formulation and also 
extended the theory for explicit consideration of hardware redundancy. 

C. Fault Detectability/Fault Discriminability 

For fault diagnosis, the ability of the sensor network to detect and discriminate failure modes and anomalous 
conditions is of prime importance. This is an extension of the observability analysis where specific state variables or 
groups of state variables and their response are indicative of health conditions. The performance metric must define 
the sensor network’s ability to distinguish these fault conditions from nominal operation (fault detection) and 
distinguish these fault conditions from each other (fault discrimination). 
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Significant research focuses on the qualitative or 
heuristic representation of the fault progression process. ( 

Research conducted by Raghuraj (ref. 31), Bhushan (ref. 

32), and Bagajewicz (ref. 22) utilized directed or sign- 
directed graphs to generate fault signatures. A directed 
graph of a hypothetical process with faults, F i? connecting to 
sensors, S i? indicating that the fault will affect the reading of 
the corresponding sensor, is displayed in figure 1 . These 
signatures provided only an indication that the sensor should 
respond to the particular fault condition, but the actual 
signature, magnitude or rate, was not utilized. Spanache 
(ref. 11) used the physical constraints of the system to 
develop a set of analytical redundancy relationships 
between the measured and unmeasured variables and 
qualitatively established an influence matrix between 
system components and possible sensor parameters. Fault 
signatures were assigned to a failed component without 
regard to specific component failure modes, and hence a 
fault signature matrix was generated between the 
component faults and the influenced analytical redundancy relationships. Narasimhan (ref. 7) used temporal causal 
graphs, and Yan (ref. 33) used a CAD system model to develop qualitative fault signatures. Representing the fault 
signature response qualitatively avoids the cost of simulating the fault in hardware and/or software and often this is 
the only level of fault information available during the early development of a system. 

In an effort to incorporate more fault information, Zhang (ref. 14) proposed a quantified directed graph (QDG) 
to model the fault propagation. For the QDG, each node represented a potential sensor with an associated signal-to- 
noise ratio, and each arc contained the fault propagation gain and direction between the sensors, along with fault 
propagation time. A sensor detectability value was computed for each sensor in the suite for a predefined set of 
faults. Fault detectability was averaged over the available sensor suite and evaluated against an assigned lower 
bound. Fault resolution was still treated somewhat qualitatively by comparing sets of sensors influenced by each 
fault. 

Azam (ref. 19) proposed a method that used a system model and reliability data to establish cause/effect 
dependencies between the faults of interest and the effects of faults on observable system parameters. The detection 
and false alarm probabilities associated with the failure source and observable discrepancy were utilized. A general 
multiple fault diagnosis algorithm was used to search for the most likely candidate fault that explained the set of 
observed discrepancies. Three diagnostic performance metrics were established for each potential sensor based on 
the decision probability error from a number of simulation runs. Selection evaluation was performed for candidate 
sensor networks by summing these performance metrics over the sensor suite and incorporating cost constraints. 

Santi (ref. 13) utilized a high-fidelity dynamic model of a rocket propulsion system to generate a simplified 
linear model. Health parameters were identified within the linear model that represented specific fault conditions. 
This research focused on mean-shift faults in the system. An inverse diagnostic model was developed which 
predicted the change in health parameters based on sensor inputs. Fault trajectories were generated and metrics were 
established for detection timeliness and fault discrimination. Fault detection required sufficient measurement 
deviation to discriminate a fault from normal operation variance. Detection thresholds were established based on the 
historical information of measurement variance, anticipated process variance and defined false alarm constraints of 
the system. Figure 2 illustrates a two-dimensional representation of the fault trajectory space where the axes 
represent sensor change from a nominal state (the origin) in increments of the sensor nominal standard deviation (a). 
The diagnostic performance metrics were based on the distance measured between fault trajectories for fault 
discrimination and inferred time to detection threshold areas for detection timeliness. In addition, fault risk reduction 
measures were incorporated into the evaluation process to allow weighting of individual failure modes based on 
criticality and occurrence rates. 



Figure 1 . — Directed graph of a hypothetical system 
displaying the propagation of possible faults (Fj) 
through the sensor (Si) network. 
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Figure 2. — Nondimensional fault trajectory space for two sensors, SI and S2. 

IV. Proposed Sensor Selection Approach 

While the design of aerospace systems/subsystems does not necessarily require an analytical methodology to 
select the location and quantity of sensors, sensor placements based solely on expert “best guess” can result in 
nonuniformity across the system and can be unverifiable especially for competing objectives such as maximizing 
safety and reducing costs. Therefore, a systematic approach based on accepted techniques and metrics is desired that 
allows for sensor network trades and consistent evaluation of metrics across the system. We direct our focus toward 
ensuring that the health assessment requirements are met by the sensor network; in that respect, several key 
questions must be addressed through evaluation of the sensor networks: 

• Can the network provide a sufficient level of detectability for the failure modes of concern? Is the sensor 
network able to detect each failure and how early after the fault is initiated? 

• Can the network discriminate between the various system failure modes? Failure discrimination need only 
be performed to the level where remediation strategies vary. 

• Can the network perform the diagnosability within the cost constraints established by the systems? This 
would include development, installation and maintenance costs, processing costs, power requirements, and 
weight. 

• How robust is the network in the presence of sensor failures? What sensors require redundancy? Are there 
common mode failures among the sensor network that could be overcome? 

In addition, it is important to utilize the maximum level of fidelity available in establishing the justification for 
sensor selection; therefore, quantitative approaches are preferred over qualitative approaches, but the methodology 
should be flexible enough to incorporate the best system design information available. The methodology selected 
should also focus on known failure modes and allow the user to incorporate weighting factors that represent both 
fault criticality and sensor costs. Finally, the methodology should incorporate other metrics, such as sensor 
reliability and sensor fault robustness, as needed. 

The approach that we have selected will advance the method described in reference 13, Systematic Sensor 
Selection Strategy (S4). This approach to sensor selection provides an architecture that fulfills the objectives 
outlined above, and a specific application of the S4 methodology to a propulsion system is provided in the appendix. 
Although the application utilizes a specific diagnostic engine, this methodology may be applicable to a wide range 
of diagnostic techniques. In addition, the architecture of this methodology should allow the implementation of other 
evaluation metrics and optimization processes like those reviewed in sections II and III. The overall objectives and 
architecture for the proposed sensor selection approach are given below. 
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A. Objectives 

The goal of the diagnostic sensor selection methodology is to provide a systematic, justifiable, creditable 
evaluation of the available sensor suite, relative to the diagnostic requirements. The methodology should be 
applicable to multiple types of systems: propulsion, electrical, hydraulic, etc. Each application and implementation 
may require unique metric algorithms, application-specific diagnostic models, and tailored optimization selection 
strategies, therefore the systematic approach should be flexible and modular. 

The methodology should be applicable across various phases of operation for the system being applied. Often 
the same system may have distinct modes of operation where diagnostic requirements may indicate unique sensor 
networks. For example, the faults and corresponding sensors required to monitor, detect, or diagnose those faults at 
one operating mode of the system (e.g., engine firing) may not be the same required at another (e.g., quiescence or 
dormant checkout phase). 

The methodology should apply throughout the design phase, regardless of the fidelity of simulations and 
domain information. As noted by both Santi (ref. 13) and Musulin (ref. 10), the fidelity of the model utilized will 
impact the performance, health assessment, or data reconciliation, and therefore the sensor suite selection process. 
Chmielewski (ref. 23) proposed that the quality of the reconciled values will depend more on the sensor 
configuration than the particular reconciliation algorithm employed. Therefore, the inputs, simulations, and fault and 
sensor information, based on “best available information” are required at each stage of the design process. Either the 
design analysis tool should demonstrate applicability throughout or define where in the process it is relevant. 

Finally, the sensor network performance depends upon both the sensor suite available and the analysis 
technique applied. An optimum sensor network established for a specific technique may not be appropriate for 
another technique. Care must be taken in generalizing the performance results across analysis functions to ensure 
appropriate solutions. The sensor selection architecture should allow different diagnostic assessment modules, 
optimization techniques, and performance metrics to be utilized. 

To that end we are proposing an architecture that will enable the following capabilities: 

• A toolbox of representative cost and performance metrics, allowing the user to specify how those metrics 
will be defined and combined. 

• Weighting capability at the component fault, sensor measurement, and performance criteria metric levels to 
allow for importance to be established by the user. 

• Established interfaces between modules to allow implementation of diagnostic modules and optimization 
techniques. A plethora of diagnostic algorithms, optimization search algorithms, and system simulations 
exist that could be utilized for this methodology. Standardization and definitions of interfaces within the 
framework will enable the inclusion of these other modules. 

B. Architecture 

Figure 3 illustrates the sensor selection architecture proposed by Santi (ref. 13) and the one that the authors have 
selected as the framework for further development and verification. The architecture can be segmented into three 
groups: Knowledge Base, Iterative Down-Select Process, and Final Selection. 

Each segment of the architecture is summarized as follows: 

• Knowledge Base — Information about the host system is gathered. If data on the host system is not available, 
experience from similar systems can provide a basis from which to define and collect pertinent health 
assessment information. The knowledge base includes the health-related information and the system 
simulation. 

♦ Health-Related Information — This information is extracted from domain experts, manufacturer reports, 
Failure Modes and Affects Analysis (FMEA), and hazard analysis studies and will include fault 
signatures, fault progressions, component/sensor reliabilities and sensor characteristics (e.g., noise). 
This data is used as inputs to the system diagnostic model, sensor suite merit algorithm, and down- 
select algorithm. In addition, it can also provide information to the system simulation. 

♦ System Simulation — Provides input in the form of data sets for the system diagnostic model and 
statistical evaluation algorithm. This module may be a collection of simulations of varying fidelities 
and applicable at varying stages of the system operation. When available and appropriate, real test or 
flight data could be provided from this module. 
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Figure 3. — Proposed diagnostic sensor selection architecture. 


• Iterative Down-Select Process — An iterative procedure to select a group of near-optimal sensor suites for 
health assessment. The process involves sequentially generating and evaluating a collection of candidate 
sensor suites, eventually converging towards a more optimum sensor suite based on the defined performance 
metric or objective function. The procedure includes a system diagnostic model, a sensor suite merit 
algorithm, and a down-select algorithm. 

♦ System Diagnostic Model — The performance of the diagnostic system is measured for each candidate 
sensor suite at each relevant operational mode. The measures of performance are provided to the 
sensor suite merit algorithm. A key feature for this model is that it must be complete and capable of 
providing diagnostic analysis for all or any subset of the possible sensors. A diagnostic model that 
must be redefined for each new sensor suite would not be appropriate for this architecture. It receives 
input primarily from the down-select module and outputs information to the sensor suite merit module. 

♦ Sensor Suite Merit Algorithm — The merit algorithm provides a score for each candidate sensor suite. It 
generates quantified metrics utilizing performance information from the system diagnostic model 
along with other pertinent characteristics of the measurement suite being evaluated. It receives input 
from the system diagnostic model and it outputs information to the down-select algorithm. 

♦ Down-Select Algorithm — The down-select algorithm is a search algorithm that utilizes an optimization 
technique employed to select sensor suites that will allow progression toward an optimal or near- 
optimal solution. Due to the large search space of the candidate sensor suites, the use of an optimized 
search algorithm, such as a genetic algorithm, allows for a timely generation of candidate sensor suites. 
The down- select algorithm receives inputs from the health-related information and sensor suite merit 
algorithm, and generates a collection of new candidate sensor suites that should “evolve” toward the 
optimum solution. 

• Statistical Evaluation Algorithm — The statistical evaluation algorithm is intended as a final robustness test 
of each candidate sensor suite. When constructing the iterative loop of the down-select process, often there 
is a trade between fidelity and speed. Therefore, a more rigorous evaluation is employed when a finite 
collection of “optimum” measurement sets has been determined. The statistical evaluation algorithm 
replicates portions of the down-select process but with additional uncertainty effects such as sensor and 
system noise characteristics, and variations in fault dynamics. Each sensor suite is evaluated by the system 
diagnostic model and the sensor suite merit algorithm. The final sensor suites can be evaluated on the 
performance with these higher fidelity simulations. The statistical evaluation algorithm uses input from the 
health-related information, system simulation data, and the collection of nearly optimal sensor suites. 
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The result of this selection process is a set of viable candidates that the designer can further assess and carry 
forward in the system design process. Each suite will have metrics measured against its performance to enable direct 
comparisons. This approach can demonstrate the impact of individual sensor measurements or to attempt to quantify 
relative benefit between sensor networks. Trade studies can be conducted to compare optional sensor networks and 
identify health monitoring gaps. Most importantly, the sensor network designer will have a valuable tool with which 
he can defend the selection made. 


V. Concluding Remarks 

The sensor selection process is the evaluation of sensor networks with the objective of fulfilling specified 
performance requirements within a set of system constraints. For the health assessment of aerospace systems, the 
performance requirements include fault detection, fault resolution, and a timely diagnosis. Other metrics can include 
gross error detectability and the ability to perform in the event of a sensor fault, as well as minimizing sensor costs. 
The paper includes a review of current literature from other disciplines (e.g., industrial and chemical processing 
applications) in order to assess the state of sensor selection technologies that might be applicable to aerospace 
systems. This investigation offers a prelude to the proposed architecture based on what is considered to be important 
criteria for a sensor selection approach for aerospace applications. A sensor selection architecture was proposed that 
will expand on the research conducted by Santi (ref. 13). The architecture’s flexibility will enable broader 
combinations of evaluation metrics to be incorporated. Further development and demonstration is required to 
advance the confidence of this approach with aerospace design applications. In addition, applicability to other 
systems beyond propulsion elements needs to be established. This will require the definition of a formal architecture 
and interfaces that will allow other modules to be easily integrated. Demonstrations need to be conducted to 
determine applicability in other areas. 
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Appendix — Systematic Sensor Selection Strategy (S4) Example 


A. Nomenclature 

Cj fault criticality weighting factor 

D rcs id detection distance metric for each training fault case 

D Total detection distance metric for the tested fault case 

Ks normalizing term for the penalty function 

A FC number of fault cases evaluated 

Nym number of tested fault cases 

N s number of sensors in evaluated suite 

As actual actual number of sensors in candidate sensor suite 

Asdesired desired number of sensors in solution sensor suite 

Pyd probability of the correct fault detection 

Pyen penalty term component of the merit function 

Ui utility cost term applied to each sensor in the current suite 

U s total utility cost associated to the current sensor suite 

Wj individual fault type weighting factor 

^penalty weight of the penalty term 

Y r recovered normalized residual sensor value 

Y t detection threshold 

Following the methodology and procedure outlined in figure 3, S4 was applied to subsystem components of the 
Space Shuttle Main Engine (SSME). While sensor selection methodologies are typically applied at the system level, 
the scope of this example was limited to facilitate presentation. More detail on this example problem along with 
entire S4 methodology is presented in the S4 User and Software Guide (ref. 34). 

As displayed in the simplified flow schematic of the SSME in figure Al, there are 3 targeted fault conditions 
and 10 candidate sensors in this example. The three fault conditions of interest in this example are a nozzle coolant 
flow leak, a high pressure oxidizer turbopump efficiency loss, and an oxidizer preburner fuel injector resistance 
increase. The candidate sensors are defined in table Al and are the only sensors considered in this exercise. 



Figure Al. — SSME flow path schematic. 
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TABLE Al.— CANDIDATE SENSORS 


Simulation variable 

Sensor description 

hpfpDP 

HPFP discharge pressure 

hpfpIP 

HPFP inlet pressure 

lpftpS 

LPFTP speed 

hpopIP 

HPOP inlet pressure 

hpotpS 

HPOTP speed 

opbPc 

OPB chamber pressure 

opovX 

OPOV actuator position 

mccOiP 

MCC OX injector pressure 

opbFiP 

OPB fuel supply duct inlet pressure 

hpotDT 

HPOT discharge temperature 


The SSME Power Balance Model was used as the simulation model because it is well established. Three 
simulation variables were identified, each representing one of the targeted fault conditions. The range of each fault 
condition, from the minimum required detection level to a maximum fault level, was defined. The complete fault 
range, from zero (nominal) to the maximum fault level, was divided arbitrarily into nine fault magnitude points. The 
SSME Power Balance Model was then used to generate simulated data for each of the 27 (3 targeted fault conditions 
for each of 9 distinct fault levels that included a nominal scenario) training case conditions. The results from all the 
simulations variables listed in table Al were recorded. A matrix of the normalized simulation inputs and results 
were then used to generate a system diagnostic model. The development of the system diagnostic model, which is 
based on by an inverse model framework, is beyond the scope of this paper. However, detailed information 
regarding the inverse model is provided in the Systematic Sensor Selection Strategy (S4) User and Software Guide 
(ref. 34). 

Once the system diagnostic model was established, simulation data at additional fault levels for the three fault 
cases were generated. The number and location of these testing cases should be selected to address diagnostic 
performance requirements properly. For this example, a single testing case was selected near the minimum required 
detection fault level for each fault scenario, in order to determine the impact of the selected sensor suite on the 
minimum detection requirements. 

From each testing case, only the values for the sensors represented within the current suite being evaluated are 
utilized. An initial detection check is made to ensure that the current suite of values exhibits detectable values at a 
minimum level. If within the current sensor suite being evaluated, the defined detection criteria are not met, then the 
suite relative to this particular fault testing case is insensitive and the fault would be undetectable. For this 
demonstration, at least one measurement had to deviate from the nominal range by 3 times its sensor noise level or 
the combined root-mean-squared-error for the entire normalized sensor suite had to be greater than 0.25, for the fault 
testing case to be detectable. If the sensor suite is determined to be insensitive, the diagnostic analysis is bypassed 
and another sensor suite is evaluated. 

If the sensor suite is sensitive to the fault case, the sensor measurement values are passed through the system 
diagnostic model along with the nine training vectors for one of the possible fault cases. The system diagnostic 
model generates the recovered hardware variable representing the potential fault case in normalized form. Since the 
system diagnostic model uses an optimization process, there will not be an exact match of the input sensor values to 
the recovered sensor values. By computing this difference for each sensor value, the system diagnostic model also 
provides the residual sensor values of the recovered fault case. This process is repeated for each of the possible fault 
cases. 

Post-processing of the system diagnostic model results provide performance parameters that can be used to 
determine the fault analysis ability of the current sensor suite. The S4 process allows the user to specify a wide range 
of metrics that are limited only in the imagination and programming ability of the user. The final performance metric 
for the sensor suite under evaluation in this example is a combination of a utility cost term, U s , the diagnosability 
performance factor, P FD and user- specified penalty term, P PE n- Each term derived for this example below. 

Ment = U s *P FD *Pj> m (1) 

For this example, the recovered sensor values were compared to the actual input sensor values. The residuals 
beyond a normalized threshold (one standard deviation for all sensors in this example) were combined into a root- 
mean squared distance value for the selected sensor suite. 
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if Y r -Y t > 0 

lv i 1 i 


( 2 ) 


if \-y t < o 

The distance value was computed for each testing fault scenario for each of the possible fault cases and 
combined into an overall metric for the sensor suite. The total distance value should reflect a preference for small 
residual distances (fault detection) when the fault input type matches the recovered fault type and large residuals 
(fault discrimination) when the fault types do not match. For each testing scenario, 

N vc 

Aotal = * Aesid/ 

z=0 

where (3) 

f 0 if input fault ^ fault(Z) 

P = 1 

[ 1 if input fault = fault(z) 

The combined merit value is then a weighted linear combination of the total distance values for each of the tested 
fault scenarios. The weighting variables allow the user to prioritize the tested fault types by importance, occurrence, 
or criticality. 



^FM 

^0 = X C /* fF /* D To,al ; (4) 

7=0 

An additional metric is incorporated in the overall sensor suite performance value that specifies a utility cost 
term (weighting factor) for the sensor involved. This utility cost term could incorporate relative weighting values 
between the sensors, such as cost, weight, power requirements, et cetera, and is simply an average of pre-assigned 
values for each of the sensors involved in the evaluated suite. 


U c = 


Ns 

r 

z=0 




Ac 


(5) 


Finally, the penalty term is given by 


P = 

1 PEN 


K, 


A n + 


(/^penalty x ASdesired — ASactual j 


( 6 ) 


As established for this example, the penalty term includes two design parameters, A N and ^penalty, which the user can 
specify initially. This term decreases from an optimal value of 1.0 as the number of sensors contained within the 
candidate suite deviates from the desired number of sensors that the user has specified. 

B. Results 

The S4 framework was executed with perturbations to parameters in the penalty term. The intent was to 
examine the number of actual sensors in the final solution sets versus the desired number of sensors selected by the 
user. The results are presented in table A2. The first column represents the desired number of sensors, the second 
column represents the weighting penalty applied and the remaining columns indicate the sensors selected from the 
S4 process. With ^penalty values of 0.0 and 0.01, the same three measurements, hpfpDp, hpotDT, and hpopIP, were 
selected regardless of the desired number of sensors specified. This measurement set scored well due to its 
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sensitivity to the targeted fault conditions permitting it to overcome the penalty term as it was applied. When the 
^penalty value was raised to 0.1, the penalty term was able to drive the solution set to the desired number of sensors 
with specific exceptions. When the target number of sensors was one, the minimal solution set contained opbPc and 
hpotDT. On the other extreme, as target sensors were increased to 9 and 10, opovX and opbFiP were never selected. 
Inclusion of these two measurements decreased the merit value more than the penalty term decreased the merit value 
for noninclusion. 

Results from this example provide motivation for more investigation. 


• As the target number of sensors increased, the same sensor set was selected again but with an additional 
sensor to satisfy the penalty term in the merit algorithm. This cannot always be expected when measurement 
information is combined in more complex examples. 

• The ability exists within the established S4 framework to force the inclusion or exclusion of one or more 
sensors from the solution set. This feature could be used to examine the sensor set that would be selected if 
one or more of the primary three sensors (i.e., hpfpDp, hpotDT, and hpopIP) were not available. 

• A more thorough analysis of the diagnostic performance results can be supported by the S4. This would 
provide design engineers the information of what system failures are covered and what failures still need to 
be addressed. 

• A statistical evaluation can be performed on the potential sensor suite candidates to determine their 
sensitivity to system variations (e.g., signal noise and engine-to-engine build variations). 


TABLE A2.— SELECTED SENSORS AS A FUNCTION OF THE DESIGNATED NUMBER OF SENSORS 

IN THE MEASUREMENT SUITE 


Nsdesired 

^penalty 

hpfpDP 

hpfpIP 

IpftpS 

hpopIP 

hpotpS 

opbPc 

opovX 

mccOiP 

opbFiP 

hpotDT 

n/a 

0 

X 





X 




X 

1 

0.01 

X 





X 




X 

2 

0.01 

X 





X 




X 

3 

0.01 

X 





X 




X 

4 

0.01 

X 





X 




X 

5 

0.01 

X 





X 




X 

6 

0.01 

X 





X 




X 

7 

0.01 

X 





X 




X 

8 

0.01 

X 





X 




X 

9 

0.01 

X 





X 




X 

10 

0.01 

X 





X 




X 

1 

0.1 






X 




X 

2 

0.1 






X 




X 

3 

0.1 

X 





X 




X 

4 

0.1 

X 




X 

X 




X 

5 

0.1 

X 




X 

X 


X 


X 

6 

0.1 

X 


X 


X 

X 


X 


X 

7 

0.1 

X 


X 

X 

X 

X 


X 


X 

8 

0.1 

X 

X 

X 

X 

X 

X 


X 


X 

9 

0.1 

X 

X 

X 

X 

X 

X 


X 


X 

10 

0.1 

X 

X 

X 

X 

X 

X 


X 


X 
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